The present invention relates to photography and more particularly to photography utilizing multi-lens panoramic cameras.
Co-pending patent application Ser. No. 09/338,790 filed Jun. 23, 1999 entitled: xe2x80x9cA System for Digitally Capturing and Recording Panoramic Imagesxe2x80x9d describe a system for simultaneously capturing a plurality of images that can be seamed together into a panorama. With the system shown in application Ser. No. 09/338,790 a sequence images can be captured to create a panoramic movie.
Co-pending patent application Ser. No. 09/310,715 filed May 12, 1999 entitled: xe2x80x9cPanoramic Movies which Simulate Movement Through Multi-Dimensional Spacexe2x80x9d describes how a views window into a sequence of panoramic images can be displayed to form a panoramic movie.
Images captured by a multi-lens camera can be seamed into panoramas xe2x80x9con the flyxe2x80x9d as they are captured and a selected view window from the panorama can be viewed xe2x80x9con the flyxe2x80x9d as the images are being captured. Such a system can be used for surveillance. Although the camera may be in a fixed position, it allows an operator to move the view window so that the operator can observe activity in any selected direction.
It is difficult if not impossible for wide-angle imaging systems to provide both wide Field of View (FOV) and high resolution. The primary benefit of spherical imaging systems is that the user can look in any direction and simultaneously view any object in reference to any other object in any direction within the environment. On the other hand, imaging systems with telephoto lenses (narrow FOV) that deliver high-resolution images can show only close-up (or narrow FOV) images without the context of wide angle or spherical views. With a high resolution system, one can, for example, read a license plate at 100 feet or recognize a human face at 100 yards. Such clarity is generally not possible when an image is captured with a wide angle lens.
It is theoretically technically possible to create a system with a large number of narrow field of view telephoto lenses which capture a large number of high resolution images which can be seamed into a spherical panorama. However, such a system would require a very large number of lenses and image sensors and it would produced billions of bytes of data every second, hence, it is generally not economically practicable to commercially build such a system.
The present invention provides an improved surveillance system which includes a multi-lens camera system and a viewer. The camera system includes a plurality of single lens cameras each of which has a relatively wide angle lens. These single lens cameras simultaneously capture images that can be seamed into a panorama that covers the entire theater (i.e. area) of interest. The camera system also includes at least one high resolution camera (i.e. a camera with a telephoto lens) that can be pointed in a selected direction to provide a high resolution image of a selected area within the theater of interest.
The wide angle lenses capture wide angle images and provide an overall view of a theater of interest and the telephoto lens provides a high resolution image of a specific area of interest. Images are simultaneously captured by the wide-angle lenses and by the telephoto lenses. The direction of the telephoto lens is controllable by a person or by a computer.
The direction of the telephoto lens relative to that of the wide-angle lenses is recorded and associated with each image captured. This allows the telephoto image to be correctly overlaid on the wide-angle image.
In some embodiments, the overlay process utilizes previously obtained calibration information that exactly maps all possible directions of the telephoto images to appropriate corresponding positions in the wide-angle imagery. In order to achieve maximum accuracy the calibration operation can be performed for each individual camera system.
In some situations, the overlay process is improved when the high resolution images are captured by the narrow FOV lens at the same time that the panorama is captured by the wide area lens. In some embodiments, the narrow FOV lens has the ability to adjust its FOV (similar to a zoom lens) under electronic control.
In one embodiment, the system displays a selected view window (i.e. a selected part) from the panorama created from images captured by the wide angle lenses. The image from the high resolution camera is superimposed or overlaid on top of the panoramic image. The higher solution image is positioned at the point in the panorama which is displaying the same area in space at a lower resolution. Thus an operator sees a relatively low resolution panorama; however, a selected portion of the panorama is displayed at a high resolution. An operator can point the high resolution camera toward any desired location, thereby providing an output which shows more detail in a selected area of the panorama. The camera is pointed in a particular direction by orienting a mirror which reflects light into the high resolution camera.
In another embodiment, the panoramic image is displayed on one monitor and the high resolution image is displayed on a second monitor (or in two windows on the same monitor). The high resolution image can also be displayed on the monitor which shows the wide angle image as in the first embodiment, or alternatively, only a cross hair or other indicia can be displayed on the first monitor to show the location of the area covered by the high resolution camera.
The synchronized high-resolution imagery integrated into or on wide-angle reference provided by the present invention can be either a still or a motion picture system.